Method and apparatus for providing rate control in a video encoder

ABSTRACT

A method and apparatus are provided for controlling the quantization level in a digital video encoder that comprises a plurality of parallel compression engines ( 12 ). The input picture is partitioned into a number of panels ( 10 ) and each panel is processed by a distinct compression engine ( 12 ). A reference quantizer scale is determined before encoding a frame of video. The reference quantizer scale is used at the first slice of every video image panel being processed by the video encoder. The quantizer scale at the last slice of the image panel is then forced to be the same as the first slice. The forcing step can use a piecewise-linear feedback formula. A group of pictures (GOP) target bit rate is adjusted based on the number of film pictures and non-film pictures currently in the processing pipeline of at least one of the compression engines. A higher target bit rate is provided for non-film pictures. A buffer ( 16 ) level of the video encoder is used to control the start of a new group of pictures (GOP). The start of a new GOP is delayed if the buffer ( 16 ) does not have sufficient space to accommodate an intra-coded (I) frame for the new GOP.

This application claims priority from U.S. provisional application No.60/102,800 filed Oct. 2, 1998.

BACKGROUND OF THE INVENTION

The present invention relates to video processing, and more particularlyto a method and apparatus for controlling the rate of data production bymultiple encoding engines provided to compress video in a televisionencoder, such as a digital high definition television (HDTV) or standarddefinition television (SDTV) encoder.

The following references are relevant to this disclosure:

-   -   1. Advanced Television Systems Committee (ATSC) A53 Standard        entitled “ATSC Digital Television Standard.”    -   2. ATSC A54 Standard entitled “Guide to the use of ATSC Digital        Television Standard.”    -   3. ISO/IEC 13818-2 “Generic Coding of Moving Pictures and        Associated Audio: Video” (MPEG2).    -   4. ISO/IEC JTC1/SC29/WG11/MPEG93/457 “MPEG2 Test Model 5.”    -   5. “Rate Control for the Grand Alliance Prototype Encoder,”        Siu-Wai Wu, memo distributed to the Grand Alliance Video        Compression Specialist Group, November 1994.

Rate control is an essential part of a video encoder. In an HDTVencoder, the picture is processed using multiple encoding engines fordata compression. These multiple encoding engines operate on the picturesimultaneously, but share a common data buffer. Thus, the rate at whichdata is produced by the multiple engines must be carefully regulated inorder to prevent buffer overflow, buffer underflow, and otherproblematic conditions.

In the past, rate control algorithms have been proposed for use inconjunction with a single video compressor. See, e.g., “MPEG2 Test Model5” referred to above. In the approach adopted by the Grand Alliance forencoding HDTV, the parallel encoding engines are synchronized at thevideo slice instead of frame level; thus, the rate control can betreated as if only a single compressor were provided. None of the priorart schemes provide a robust solution to the problem of regulating therate of data production by a plurality of compressors, in order tocontrol the quantization of a digital video encoder that uses parallelcompression engines.

It would be advantageous to provide an efficient rate control algorithmto regulate the rate of data production by multiple encoding engines tooptimize video quality. It would be further advantageous to implementsuch an algorithm in the video encoder's existing controllermicroprocessor. The present invention provides a method and apparatusenjoying these and other advantages.

SUMMARY OF THE INVENTION

In accordance with the present invention, rate control is provided in avideo encoder having multiple compression engines that process a picturesimultaneously and share a common buffer. Nonlinear feedback is used toforce a quantizer scale at the last slice of an image panel to beapproximately the same as the quantizer scale at the first slice of theimage panel. This avoids visible artifacts caused by abrupt changes inthe quantizer scale at the panel boundaries.

A target bit rate for a group of pictures (GOP) is adjusted based on thenumber of film pictures and non-film pictures currently in theprocessing pipeline of the compression engines. Since non-filmprocessing in general requires a higher target bit rate, this adjustmentallows the rate control algorithm to adapt when the video switches fromfilm to non-film or vice versa.

The buffer level is used to control the start of a new GOP. If there isnot sufficient space in the buffer to accommodate the intra-coded (I)frame, the start of a new GOP is delayed.

More particularly, methods and apparatus are provided for controllingthe quantization in a digital video encoder that comprises a pluralityof parallel compression engines. A target quantization level isdetermined for a video frame. The target quantization level is appliedto the first slice of every video image panel of a plurality of panelsrepresenting the video frame and being processed in parallel by thevideo encoder. The quantization level is driven toward the targetquantization level at the last slice of each of the image panels.

The driving step can use, for example, piecewise linear feedback todrive the quantization level of the last slice of each of the imagepanels toward the target quantization level. The feedback can compensatefor variations in the quantization level between the first and lastslice of each image panel.

A group of pictures (GOP) target bit rate can be adjusted based on anumber of film pictures and non-film pictures currently in a processingpipeline of at least one of the compression engines. A higher target bitrate can be provided for non-film pictures.

A buffer level of the video encoder can be used to control the start ofa new group of pictures (GOP). The start of a new GOP is delayed ifthere is not sufficient space in the buffer to accommodate anintra-coded (I) frame for the new GOP.

Every input picture can be partitioned into a fixed number of horizontalpanels, wherein each of the panels is compressed by a separatecompression engine. The panels can be simultaneously compressed during aframe time. The compressed panel data are stored locally at thecompression engines for subsequent transfer to a video buffer of thevideo encoder within a next frame time. In order to form a transportpacket, data are retrieved from the buffer at an average rate equal to aspecified video bit rate whenever the buffer has at least one transportpacket payload's worth of data. Null packets are substituted for videopackets to maintain a constant transport bit rate whenever the bufferlevel falls below one transport packet payload's worth of data.

A reference quantizer scale is calculated for each of the compressionengines. The reference quantizer scale is calculated based on anaccumulation of quantizer scale values for the compression engine(PanelSumQuant), an accumulation of the number of bits generated on thatcompression engine (PanelBitCount); an accumulation of the number ofmacroblocks processed on that compression engine (PanelMbCount); and afullness level of a video buffer of the video encoder(VideoBufferLevel). The compression engine modifies its referencequantizer scale based on a local buffer fullness to generate a finalquantizer scale value for use in quantization. A panic mode can beinitiated by the compression engine if the final quantizer scale valueis higher than a predetermined maximum value. The panic mode forces allor part of the DCT coefficient to take on a zero value.

A method is also provided for controlling the rate at which data isprocessed by a digital video encoder using a plurality of parallelcompression engines to compress successive macroblocks of video data.Frame level rate control is provided for successive video frames inaccordance with a picture start interrupt service routine once per codedpicture. Macroblock level rate control is provided in accordance with arate control interrupt service routine a plurality of times per codedpicture.

The picture start interrupt routine updates frame level statisticalvariables by processing data collected from a prior coded frame. Atarget bit rate is calculated for a new group of pictures (GOP) if a newframe to be processed comprises an intra-coded (I) frame. For everyframe to be encoded, upper and lower limits are calculated for thenumber of bits that a new frame is allowed to generate. A target numberof bits is computed for each frame and for each of a plurality of panelsinto which each frame is divided. The target number of bits can becomputed by distributing the bits in proportion to relative complexityvalues of the video frames being processed. A frame target quantizerscale is calculated from the target number of bits computed for a frame.The frame target quantizer scale is provided to each of the plurality ofcompression engines as an initial reference quantizer scale for use incommencing coding of the video frame.

In a preferred embodiment, the compression engines can force some of theDiscrete Cosine Transform (DCT) coefficients of the video framemacroblocks to zero, if the reference quantizer scale is too high tomaintain the number of bits that the new frame is allowed to generatewithin the applicable upper limit.

The rate control interrupt service routine provides interrupts whichinitiate a modulation of a frame target quantizer scale. The modulationcan be based on feedback to attempt to bring the actual number of bitsfor each of the video frame panels to the corresponding target number ofbits for the panel.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 is a block diagram illustrating an HDTV encoder with which thepresent invention may be utilized.

DETAILED DESCRIPTION OF THE INVENTION

The present invention provides a method and apparatus for rate controlin a digital video encoder that uses multiple compression enginesrunning in parallel. In accordance with a preferred embodiment of theinvention as illustrated in FIG. 1, every input picture is partitionedinto eight horizontal panels by an HDTV panel splitter 10. Each panel isthen compressed by an individual video panel compressor 12, for example,the DvXpert chip available from C-Cube Microsystems of 1778 McCarthyBlvd., Milpitas, Calif. 95035, USA. It should be appreciated that adifferent number of panels can be utilized, and that the invention canbe implemented on video encoders other than the DvXpert encoderdescribed herein.

The panels into which the input pictures are partitioned aresimultaneously compressed during a frame time. The compressed panel dataare stored locally on the panel compressor, and then transferred to theprimary video FIFO 16 (“video buffer”) within the next frame time.Whenever the video FIFO 16 has at least one transport packet payload'sworth of data (184 bytes), the bits are retrieved from the FIFO by apacket processor 18 at an average rate equal to the user specified videobit rate (bit_(—)rate). The retrieved bits are used to form a transportpacket in accordance with the well known MPEG (Motion Picture ExpertsGroup) standard. Whenever the FIFO level falls below 184 bytes, nullpackets will be substituted for video packets to maintain a constanttransport bit rate, effectively reducing the video bit rate. Asillustrated in FIG. 1, packet processor 18 outputs the compressedbitstream.

The rate control function is mainly performed by a master compressioncontroller (MCC) 14. The MCC collects the statistics (PanelSumQuant,PanelBitCount, and PanelMbCount) from the panel compressors and thebuffer level from the video FIFO, and then calculates an 8 bit referencequantizer scale (PanelQuant) for each panel compressor. The panelcompressor then modifies the reference quantizer scale based on localactivities and local buffer fullness to generate the final quantizerscale value to use for quantization. If the desired quantizer scale ishigher than the maximum supported by the MPEG quantization table, thepanel compressor may apply a panic mode and force some of the DCT(Discrete Cosine Transform) coefficients to zero.

A sync interrupt generator 20, which receives the input HDTV video,generates a field sync interrupt to the MCC at the beginning of everyinput field, and a Rate-Control (RC) Sync interrupt periodically (e.g.,every 1.67 ms for 1080I mode) within a field time. A field syncinterrupt is qualified as a picture start if the panel compressors areready to encode a new picture at the current field time. Rate control bythe MCC is performed in two parts: (1) Frame level rate control iscarried out at the Picture Start interrupt service routine once percoded picture; (2) Macroblock level rate control is carried out at theRC interrupt service routine up to 20 times per coded picture for 1080line interlaced (1080I) mode (up to 10 times per coded picture for 720line progressive scan (720P) mode). On average, only about eight RCinterrupts are used to regulate the quantization because the panelcompressor only uses a portion of the field time to carry out theencoding process.

The goal of rate control is to maintain a consistent video quality overtime and over the picture. A basic assumption of the rate controlalgorithm is that the coding complexity of a particular picture type isapproximately constant from frame to frame of a steady scene, and ismeasured by the product of the average quantizer scale and the number ofbits generated over the frame (or a panel). The complexity of thecurrent frame is normally estimated from that of the previous frame,except a when a scene change or fade-in from black is detected, in whichcase the complexity estimates are reset to some provisional values.

At picture start interrupt, the MCC first updates all frame levelstatistical variables by processing the data collected from the lastcoded frame. A target bit rate for the new GOP is calculated if theupcoming frame is an intra-coded frame (I frame). The GOP target bitrate calculation aims to bring the encoder buffer to a consistently lowlevel at the beginning of every GOP so that there is enough room in thebuffer to accommodate the surge of bits produced by the new I frame.

For every frame to be encoded, the MCC calculates the upper and lowerlimits on the number of bits that the new frame is allowed to generate.These limits are based on the constraint that neither the encoder buffernor the decoder buffer is allowed to overflow nor underflow. The MCCalso computes the target number of bits to be generated for the frameand for each panel by distributing the bits in proportion to therelative complexity values. Then, a frame target quantizer scale iscalculated from the target frame bit rate. The frame target quantizerscale is delivered to every one of the multiple panel compressors as theinitial reference quantizer scale to start coding the frame.

At each Rate Control Sync interrupt, the MCC modulates the frame targetquantizer scale by a feedback mechanism which tries to bring the panelbit count to a value that equals the targets. If the accumulated bitcount is approaching the upper limit that is calculated at the picturestart, the modulated quantizer scale is further increased to slow downthe production of bits. The resulting reference quantizer scale value isthen clipped to fit into an eight bit integer value for delivery toevery panel compressor.

Upon receiving the reference quantizer scale, the panel compressorchecks its local buffer level and increases the reference quantizerscale if needed to protect the buffer. The panel compressor may alsoforce some DCT coefficients to be zero if the reference quantizer scaleis too high. Then, adaptive quantization is performed in a mannersimilar to the MPEG Test Model 5 and the actual quantization scale islooked up from the Moving Picture Experts Group (MPEG) quantizationscale table.

The following sections detail the operation of the rate controlalgorithm.

1. List of variables Default Name Value Description BitRate UserElementary video bit rate (bits per define second) N User Nominal GOPLength define M User Temporal distance between define consecutive I or Pframes Nb N/A Nominal number of B frames in a GOP Np N/A Nominal numberof P frames in a GOP FrameRate User Frame rate of the input video definePanelSumQuant[0 . . . 7] N/A Accumulation of quantizer scale values oneach panel compressor PanelBitCount[0 . . . 7] N/A Accumulation ofnumber of bits generated on each panel compressor AvgQuant N/A Averagequantizer scale value of the frame just coded SumPanelBitCount N/A Totalnumber of bits generated by all panel compressors combined for the framejust coded. In RC sync, this is the number of bits generated so far forthe frame. K[I, P, B] 0.75, Weighting of I, P and B frames 1.0, 0.75X[I, P, B] N/A Estimate of the coding complexity StartupX[I, P, B] N/AStart up value of the coding complexity estimate. StartupQuant N/AApproximate minimum quantizer scale value to use for startup or scenechange or fade in. HeaderBitCount N/A Number of bits in the pictureand/or sequence header(s) of the frame just encoded. FrameBitCount N/ATotal number of bits generated by the frame just encoded.VideoBufferLevel N/A Number of bits stored in the physical video FIFOEncoderBufferLevel N/A Total number of bits store in the encoder'sbuffers combined BufferDelay N/A Total buffer delay of the encoder anddecoder combined. TargetBufferLevel N/A Approximate level of encoderbuffer at the start of a new GOP GopDuration N/A Estimate duration ofthe current GOP PipelineSize N/A Number of frames in the encoder'scurrent processing pipeline PipelineRepeatFields N/A Number of repeatedfields (3:2 pulldown) in the encoder's current processing pipeline.PictureDuration N/A Estimated duration of the current frame. β 0.75 GOPfeedback factor MaxJitter 0.004 Maximum transport jitter in MPEG secondstandard FrameMaxDecoder N/A Maximum number of bits the current frame isallowed to generate, as constrained by decoder buffer. FrameMaxEncoderN/A Maximum number of bits the current frame is allowed to generate, asconstrained by encoder buffer. VideoFifoSize 125829 Size of encoder'sphysical video 12 bits FIFO FrameMaxBits N/A Maximum number of bits thecurrent frame is allowed to generate, as constrained by both encoder anddecoder. InitialFrameTargetBits N/A Target number of bits to begenerated by the current frame, without feedback. FrameTargetBits N/ATarget number of bits to be generated by the current frame.AllocationFactor[P, B] N/A Ratio of bit allocation to P and B frames.PictureFeedback N/A Feedback of surplus of deficit of bits at picturelevel. α N/A Picture feedback factor AccumulatedFrameTar N/AAccumulation of frame target bits getBits since the beginning of thecurrent GOP FrameMinTarget N/A Minimum number of bits to generate toprevent the encoder video FIFO from emptying. MaxTarget[I, P, B] N/AMaximum allowable value of FrameTargetBits MinTarget[I, P, B] N/AMinimum allowable value of FrameTargetBits FrameQuant N/A Targetquantizer value for the frame MinFrameQuant 2 Smallest target quantizervalue for the frame LastFrameQuant N/A Target quantizer value of thelast coded frame of the same picture type LastAvgQuant N/A Average ofthe actual quantizer scale of the last coded frame of the same picturetype. PartialFrameTargetBits N/A Portion of the FrameTargetBits asscaled by the number of macroblocks coded so far in the frame.PartialFrameMaxBits N/A Portion of the FrameMaxBits as scaled by thenumber of macroblocks coded so far in the frame. SumPanelMbCount N/ANumber of macroblocks coded so far in the frame by all the panelcompressors combined CurrentQuant N/A The modified quantizer scale valuecurrently being used. MacroblocksPerFrame N/A Number of macroblocks in aframe. MaxRcQuant 144 Maximum quantizer scale value without invokingpanic mode. MaxQuantNonIntraDC N/A DC entry of the non intra Mulquantization muliplication table for quantizer scale value of 112.PanicNonIntraDCMul N/A DC entry of the non intra panic tablePanicThreshold1, {750000, Thresholds of FrameMaxBits forPanicThreshold2, 500000, Panic mode. PanicThreshold3 1500000} for P andB frames; {1500000, 1000000, 3000000} for I frame LocalBufferSize 345216Size of the panel compressor's 0 bits local buffer LocalBufferLevel N/AFullness of the panel compressor's local buffer PanelQuant N/A Actualquantizer scale value used by the panel compressor2. Initialization

-   1) Set all parameters to default values according to Table 1 and    Table 2.-   2) The Packet Processor sends the data packet from the video FIFO as    soon as the first 184 bytes of the compressed data packet are    written into the FIFO.    3. Configuration

The rate control task receives the following configuration parameters:

-   -   1. Elementary video stream bit rate (BitRate).    -   2. Nominal GOP length (N).    -   3. Temporal distance between every pair of I/P frames (M).    -   4. Video Format (1920×1080I or 1440×1080I or 1280×720P).    -   5. Video frame rate (29.97 Hz or 30 Hz or 59.94 Hz or 60 Hz).        4. MCC Picture Start Interrupt        A) Update Complexity Estimate from the Last Coded Frame

First, the frame bit count and average quantizer scales are calculatedfrom the data read from the panel compressors. The bit count used in thecomplexity calculation is only for the slice layer bits. The bitsgenerated for the headers, which are generated by the MCC, are notincluded.

-   -   AvgQuant=sum of PanelSumQuant over all 8 panels/number of        macroblocks per frame    -   SumPanelBitCount=sum of PanelBitCount over all 8 panels        These statistics are used to calculate the coding complexity of        the previous coded picture as:        AvgQuant*SumPanelBitCount

It is assumed that the coding complexity for a particular picture typeis relatively constant over a steady scene content, so that thecomplexity of a frame yet to be encoded may be estimated from the actualcoding complexity of the most recently coded frames of the same picturetype. Since I frames occur infrequently, only the most recently coded Iframe is used as the complexity estimate (X[I]). As a compromise betweenstability and response speed, four most recently coded frames of thesame picture type are used to generate the complexity estimate for P andB frames. It should be appreciated that averaging more frames provides amore stable complexity estimate but a slower response time. Likewise,averaging fewer frames provides a less stable complexity estimate but afaster response time.

The rate control algorithm maintains a queue to store the complexityvalues of the four most recently coded P frames, and a queue to storethe complexity values of the four most recently coded B frames. Thecomplexity estimate for P (or B) frame, X[P] (or X[B]), is thencalculated as the average of the four complexity values stored in the P(or B) complexity queue.

The complexity estimates for I, P and B frames are used to calculate thetarget quantizer scale value and the bit budget for upcoming frames tobe encoded. Details are described in the following sections.

Since the complexity estimates are computed from statistics generatedfrom encoding the past frames, special handling is required at start up,scene change, or fade to reset the complexity estimate when the codingcomplexity from the past is not available or is not reliable.

At startup, the I complexity estimate and the P and B complexity queuesare initialized to a set of default startup complexity values(StartupX[I/P/B]). If the current frame to be encoded is the first I orP frame after fade-in from black or scene change, the I complexityestimate and the values stored in the P and B complexity queues areconditionally replaced by the default startup value. Every stored valuein the P and B complexity queue and the I complexity estimate, arecompared with the startup complexity. The stored complexity is replacedby the startup complexity if the stored complexity is smaller.

If multiple scene changes occur within proximity to each other (e.g.flashes, special effects, or a bad edit), the complexity estimates arereset only once at the first scene change. This is achieved by skippingthe reset if the complexity adjustment occurred in the previous P frame.

The default startup complexities are moderate complexity valuescalculated based on the assumption that an I frame consumesapproximately 4 times as many bits as a P frame, and B frames consumeapproximately 20% fewer bits than P frames. This ratio of 4.0:1.0:0.8 isobtained from empirical observations. Other values of I:P:B ratios arepossible. Furthermore, the startup complexities assume a provisionalaverage quantizer scale value (StartupQuant) which varies linearlybetween 20 to 28 for bit rate values between 10 Mbps to 26 Mbps. Thisprovisional average quantizer value is clamped at 20 for bit rate valueslower than 10 Mbps, and at 28 for bit rate values higher than 26 Mbps.

The startup complexity values for I, P, and B frames are calculated asfollows:

StartupX [I] = 4.0 * StartupQuant * N * (BitRate / FrameRate) / (4.0 +1.0*Np + 0.8*Nb) StartupX [P] = 1.0 * StartupQuant * N * (BitRate /FrameRate) / (4.0 + 1.0*Np + 0.8*Nb) StartupX [B] = 0.8 * StartupQuant *N * (BitRate / FrameRate) / (4.0 + 1.0*Np + 0.8*Nb)To accelerate the convergence of the P and B complexity estimate to thenew scene, a state machine is implemented to handle the complexityupdate for the P and B frames following the complexity reset.

-   -   1. For the initial I or P frame after scene change (or fade-in        from black), the complexity estimates are set to the reset value        before the frame is encoded. After encoding the initial frame,        the complexity estimates are updated using the new encoding        complexity (AvgQuant*SumPanelBitCount). If the initial frame is        an I frame, the I complexity estimate is updated. If the initial        frame is a P frame, the entire P complexity queue is replaced by        the new encoding complexity. The P encoding complexity is used        to estimate the B encoding complexity, so the entire B        complexity queue is replaced by the encoding complexity of the P        frame multiplied by the value 0.8, which value is the estimated        ratio of B complexity to P complexity. Other reasonable values        can, of course, be used.    -   2. Because of reordering, the first group of B frames following        the initial I or P frame may belong to the old scene. Therefore,        if there is only one B frame in the group (M=2), the encoding        complexity after encoding that B frame is discarded so that the        B complexity queue stays at the reset value after that B frame        is encoded. If there is more than one B frame in the first group        of B frames (M=3), the encoding complexity of the first B frame        is used as the complexity estimate for the next B frames in the        group. Therefore, the B complexity queue is updated normally        except for the last B frame in the group. After encoding the        last B frame in the group, the B complexity queue is set to the        reset value so that statistics possibly from the old scene will        not contaminate the complexity estimates of the new scene.    -   3. If the initial frame is an I frame, after encoding the first        P frame that follows the initial frame, the entire P complexity        queue is replaced by the new encoding complexity. The B        complexity queue is replaced by the encoding complexity of the P        frame multiplied by a value such as 0.8, i.e., an estimated        ratio of B complexity to P complexity. If the initial frame is a        P frame, the complexity queue is updated normally after encoding        the first P frame following the initial frame.    -   4. After encoding the first B frame in the second group of B        frames following the initial frame, the entire B complexity        queue is replaced by the new encoding complexity        (AvgQuant*SumPanelBitCount) of the B frame.    -   5. The complexity queues are updated normally (by replacing the        oldest entry by a new value) for every frame thereafter. For        example, consider a scene change that occurs at frame I0 in the        following sequence of frames (in encoding order): I0, B1, B2,        P3, B4, B5, P6, . . . . Before encoding I0, the complexity        estimates are forced to reset. The I complexity estimate is        updated after encoding I0. The B complexity queue is set to the        new encoding complexity after encoding B1. This new complexity        estimate is used for encoding B2. After encoding B2, the B        complexity queue is set to the reset value. After encoding P3,        the entire P complexity queue is replaced by the encoding        complexity of P3, and the entire B complexity queue is replaced        by the same value multiplied by, e.g., 0.8. After encoding B4,        the entire B complexity queue is replaced by the encoding        complexity of B4. From B5 and P6 onward, the complexity queues        are updated normally.

Consider a case where a fade-in from black happens at frame P0 in thefollowing sequence of frames (in encoding order): P0, B1, B2, P3, B4,B5, P4, . . . . Before encoding P0, the complexity estimates (includingI complexity) are forced to reset. After encoding P0, the P complexityqueue is replaced by the encoding complexity of P0, and the B complexityqueue is set to the same value multiplied by, e.g., 0.8. The Bcomplexity queue is set to the new encoding complexity after encodingB1. This new complexity estimate is used for encoding B2. After encodingB2, the B complexity queue is set to the reset value. After encoding P3,both P and B complexity queues are updated in the same way as afterencoding P0. After encoding B4, the entire B complexity queue isreplaced by the encoding complexity of B4. From B5 and P6 onward, thecomplexity queues are updated normally.

A fade-out to black increases the complexity for P and B frames butdecreases the complexity of I frames. Therefore, on the first frame whena fade-out is detected, a constant factor larger than one is multipliedwith every entry of the B and P complexity queues to increase thecomplexity estimate. For example, the multiplier 1.25 can be used for Pcomplexity and the multiplier 1.5 can be used for B complexity. The Icomplexity estimate is reduced by multiplying a constant factor lessthan one for every frame where fade-out is detected. The multiplier0.975 can, for example, be applied on I complexity for every frameduring a fade-out to black.

B) Update the Encoder's Buffer Status

Since the data transfer rate of the compressed data from the panelcompressors' local buffers to the video FIFO via the PCI bus is muchfaster than the data output rate from the video FIFO, we can considerthe combined video FIFO and the panel compressor's local bufferstogether as a single entity for the encoder's buffer levelconsiderations. The level of the video FIFO is latched at Field Sync andis read by the MCC via the PCI bus interface.

First the total number of bits generated from the previous frame iscalculated as follows:FrameBitCount=SumPanelBitCount+HeaderBitCountThen, the total encoder buffer level is computed as:EncoderBufferLevel=VideoBufferLevel+FrameBitCount

A new GOP is set up if the current frame is an I frame. Before encodingan I frame, the MCC rate control algorithm calculates the bit budget forthe new GOP, assuming a nominal number of frames for the GOP (N). TheMCC rate control algorithm tries to bring the video buffer to a constantlow level (TargetBufferLevel) at the start of every GOP. TheTargetBufferLevel is set to be a fraction (one-eighth) of the usablebuffer space in the encoder and decoder combined.TargetBufferLevel=(BufferDelay*BitRate)/8

-   -   where BufferDelay is the total buffer delay which is detailed in        section D below.

First, the MCC estimates the time duration of the upcoming GOP(GopDuration) based on the film mode status. The GOP is divided into twoparts, namely, (i) the frames already preprocessed and waiting in theencoder's pipeline, and (ii) the future frames that have not yet enteredthe encoder's pipeline. For the frames that are already in the pipeline,the exact duration of the frames can be calculated from the number offrames in the pipeline (PipelineSize) and the number of frames that havea repeat field (PipelineRepeatFields). For the future frames that havenot yet entered the pipeline, it is assumed that those frames have thesame film mode classification as the most recently preprocessed frame,and the three-two pulldown pattern is continued if they are classifiedas film.

The current I frame is not included in the calculation of thePipelineSize and PipelineRepeatFields. However, the exact duration ofthe current frame (PictureDuration) is known from the repeat fieldstatus. If the current frame to be encoded has repeat first field, thePictureDuration equals (1.5/FrameRate), otherwise, PictureDurationequals (1.0/FrameRate). This process can be implemented using thefollowing code:

if (most recently preprocessed frame is film){ if ((N − 1 −PipelineSize) is even){ GopInputFrames = PipelineSize + (0.5 *PipelineRepeatFields) + 1.25 * (N − 1 − PipelineSize) } else { if (mostrecently preprocessed frame has repeat first field){ GopInputFrames =1.0 + PipelineSize + (0.5 * PipelineRepeatFields) + 1.25 * (N − 2 −PipelineSize) } else { GopInputFrames = 1.5 + PipelineSize + (0.5 *PipelineRepeatFields) + 1.25 * (N − 2 − PipelineSize) } } } GopDuration= PictureDuration + (GopInputFrames/InputFrameRate)

The target bit rate of the new GOP is the average number of bitstransmitted during the GOP plus the surplus or deficit carried over fromthe previous GOP. This carry over is reflected by the deviation of thecurrent buffer level from the target buffer level. In particular:GOP_(—)target=(GopDuration*BitRate)+β*(TargetBufferLevel−EncoderBufferLevel)where β is an adaptive feedback control factor, such that β=1.0 whenthere are surplus bits carried over from the previous GOP(EncoderBufferLevel<TargetBufferLevel) and β=0.75 when there is adeficit (EncoderBufferLevel>TargetBufferLevel). This arrangement allowsthe rate control algorithm to respond quickly to make use of the unusedbit (if any) left over from the previous GOP, yet avoids starving thenew GOP if the previous GOP consumes more bits than the budget. Thus,any negative feedback is absorbed by the buffer and spread over futureGOPs to avoid sudden degradation in the video quality. The GopTarget maybe adjusted later during the GOP if the GOP size is extended to alignthe next I frame with a scene change, or if the film mode statuschanges.C) Adjust the GopTarget

Within a GOP, the GOP target is adjusted every frame for the following:

-   -   i) If a scene change is detected, the encoder may lengthen the        GOP to align the next I frame with the scene change. In this        case, the GopTarget is increased accordingly to provide the bits        needs for coding the extra bits. This adjustment is only done on        P frames, since extra B frames must be accompanied by P frames.        If the current frame is film, the increment is        (1.25*M*BitRate/FrameRate), otherwise the increment is        (M*BitRate/FrameRate). The constant M is the distance (in        frames) between two consecutive P frames.    -   ii) When the initial target bit budget is calculated at the        beginning of the GOP, it is assumed that the entire GOP will        operate in film mode if the last preprocessed frame is film, or        in regular video mode if the last preprocessed frame is        non-film. However, the film mode status may change in the middle        of a GOP.

If the most recently preprocessed frame belongs to the current GOP andswitches from non-film to film, the GopTarget is increased accordinglyto account for the expected increase in the expected duration of theGOP. The increase in GopTarget is calculated as:0.25*(BitRate/FrameRate)*(Number of unprocessed frames remaining in theGOP),where the number of unprocessed frames is calculated as:N−(Number of frames already encoded in the GOP)−PipelineSize−1.

On the other hand, if the input switches from film to normal video, theGopTarget is decreased by the same amount to account for the expecteddecrease in the duration of the GOP. The film to video (or video tofilm) adjustment is skipped if the number of frames encoded so far inthe GOP exceeds (N−PipelineSize−1).

D) Buffer Protection

The end-to-end buffer delay is set by the packet processor and isdefined as the time elapsed from the time the first bit of the pictureenters the encoder FIFO to the time the picture is pulled out from thedecoder FIFO (Decoder Time Stamp (DTS) of the picture). The MCC ratecontrol algorithm uses the delay value to calculate the maximumallowable number of bits in the picture that does not overflow theencoder buffer or underflow the decoder buffer. For high bit rates, thesystem stores as many bits as the decoder buffer allows to maximallyutilize the buffers. For low bit rates, however, an upper bound of,e.g., one-half second delay is set to ensure a reasonable channelacquisition delay. For compliance with the ATSC standard, it is assumedthat the decoder uses a 488*1024*16=7995392 bit buffer. The buffer delayis calculated from the following equation:BufferDelay=Min (7995392 bits/BitRate, 0.5 second)The delay is fixed at, e.g., 0.5 second for bit rates below 16 Mbps, anddecreases linearly for bit rates above 16 Mbps, e.g. 0.2 second at 40Mbps.

To prevent the decoder buffer from underflowing, all the bits currentlyin the encoder buffer plus the bits to be generated for the currentframe have to be transported to the decoder before the decode time ofthe current frame. Since the PCI bandwidth is much higher than theoutput bit rate, we can assume that the transfer of data from the panelcompressors to the video FIFO is instantaneous. The worst case decodetime of the current frame is (BufferDelay−MaxJitter) time units aftercurrent picture is transferred to the video FIFO, which happens onepicture processing time later.

If the encoder's current buffer level is higher than the nominal numberof bits (BitRate*PictureDuration) to be transmitted from the encoderbuffer to the decoder buffer during the current frame time, a constantoutput bit rate (BitRate) will be maintained. Otherwise, the packetprocessor will generate null packets to fill up the gap in the transportbandwidth, which effectively reduces the video bit rate. Therefore, theprojected video FIFO level one frame time later when the current pictureenters the video FIFO is:max (EncoderBufferLevel−(BitRate*PictureDuration), 0)From the time the current frame enters the video FIFO to the decode timeof the frame, worst case (BufferDelay−MaxJitter)*BitRate bits will betransported to the decoder FIFO so that the number of bits available forthe current frame is:FrameMaxDecoder=(BufferDelay−MaxJitter)*BitRate−Max(EncoderBufferLevel−(BitRate*PictureDuration), 0)

To prevent the encoder's video FIFO from overflow, the size of thecurrent frame plus the level of the FIFO when the current frame istransferred to the video FIFO should not exceed the size of the videoFIFO (VideoFifoSize), i.e.:FrameMaxEncoder=VideoFifoSize−MaxEncoderBufferLevel−(BitRate*PictureDuration), 0)In an example implementation, the size of the video FIFO is 12582912bits, although any other suitable FIFO size can be used as will beappreciated by those skilled in the art.

The maximum allowable bitstream size of the current frame is thencalculated as the smaller of FrameMaxEncoder and FrameMaxDecoder. Anoffset of two transport packets' worth of video data (2*8*184=2944 bits)is included to provide a safety margin. Thus:FrameMaxBits=MIN (FrameMaxEncoder, FrameMaxDecoder)−2944

Since the packet processor inserts null packets to effectively reducethe video bandwidth when there is not enough data in the video FIFO, itis not necessary to set a lower bound on the frame size to protect theencoder or decoder buffer.

E) Calculate Bit Budget for the Frame

For every picture to be encoded, the MCC rate control algorithmcalculates a target for the number of bits to be generated for the frame(FrameTargetBits). The basic idea is to distribute the available bits tothe I, P, and B frames in proportion to the complexity estimatesadjusted by a set of constant weighting factors (K[I], K[P], K[B]). Theweighting factors (K[ ]) account for the different perceptual andstatistical characteristics of I, P and B frames. As an example, theweighting factors (0.75, 1.0, 0.75) can be used, although otherweighting factors are equally applicable. A portion of the deviation ofthe actual bits used from the target is fed back to adjust the targetfor the future frames.

For an I frame:FrameTargetBits=GopTarget*K[I]*X[I]/(K[I]*X[I]+Np*K[P]*X[P]+Nb*K[B]*X[B])The smaller of FrameTargetBits value of the I frame and the actualnumber of bits generated by the I frame is then deducted from theGopTarget before allocating the remaining bits to the P and B frames.Note that if the actual number of bits generated exceeds the target, thetarget is deducted from the GopTarget instead of the actual bits becausethe I frame consumes bits at a much higher rate than P and B frames.Deviation of the I frame's bit consumption from the target may depletethe bit budget of the remaining P and B frames. By decoupling the actualI frame bits from the B and P bit allocation, the feedback is deferredto the next GOP.

For B and P frames, an allocation ratio is first calculated as follows:AllocationFactor[P]=(N−1)/(Np+Nb*Max (½, Min (3,(K[B]*X[B]/K[P]*X[P]))))AllocationFactor[B]=(N−1)/(Nb+Np*Max (⅓, Min (2,(K[P]*X[P]/K[B]*X[B]))))The ratio of the weighting complexity (K*X) of P to B frames are boundedby ½ and 3 to avoid extreme allocation caused by an inaccuratecomplexity estimate. An initial frame target is then computed byapplying the allocation factor to the average number of bits availableto the P and B frames, as follows:InitialFrameTargetBits=AllocationFactor*GopTarget/(GopLength−1)Note that GopLength is the current length of the GOP, which may belarger than NominalGopLength because the GOP is extended to align thenext I frame with the scene change. If the current frame count withrespect to the start of the GOP is less than the nominal GOP length (N),GopLength is set to the same value as N. If the current frame countexceeds N, GopLength is increased by M frames for every time a P frameis encountered. Feedback is then added to the InitialFrameTargetBits tocompensate the deviation of the actual bit consumption from the target.The feedback is computed as:PictureFeedback=*(AccumulatedFrameTargetBits−GopBitCount)

AccumulatedFrameTargetBits is the accumulation of the FrameTargetBitssince the beginning of the GOP. The PictureFeedback portion of theFrameTargetBits is excluded in the accumulation in every P and B frame.GopBitCount is the number of bits generated by the current GOP up to andincluding the last coded frame.

After encoding each frame, the GopBitCount is increased by the effectivenumber of bits generated by the frame. Since the null packet is insertedto sustain the transport bit rate when the video FIFO level is lowerthan that required to produce one transport packet, the effective numberof bits produced for the last coded frame is lower bounded by the numberof bits that are required to sustain a constant output bit rate duringthe upcoming frame time minus the number of bits that are already in thevideo FIFO. The effective number of bits generated is:Max (FrameBitCount, (PictureDuration*BitRate−VideoBufferLevel)

The constant α controls the response of the system. If α is too big, thesystem becomes unstable; if α is too small, the response is slow. In anexample embodiment, α can be set to 1/N, although other values can besubstituted.

To avoid excessive negative feedback that may deplete the bit budget ofthe current frame, the magnitude of PictureFeedback is bounded to notexceed one-third of the InitialFrameTargetBits if the feedback isnegative.FrameTargetBits=InitialFrameTargetBits+PictureFeedback

Finally, the FrameTargetBits is bounded both from above and below. It isnecessary that the FrameTargetBits does not lie in the proximity of themaximum allowable number of bits that applies for buffer protection.Otherwise, panic mode may be invoked too early and the picture qualitywill be excessively degraded. Although it is not necessary to set aminimum picture size to protect the buffer, it is desirable that theencoded frame is big enough such that bandwidth would not be wasted bysending null packets. Therefore, a minimum value for the FrameTargetBitsis calculated as the worst case number of bits required to prevent theencoder video FIFO from emptying.FrameMinTarget=(1.5*BitRate/FrameRate)−Max(EncoderBufferLevel−(BitRate*PictureDuration), 0)To avoid extreme values that may result from inaccurate complexityestimates, hard limits are provided so that the FrameTargetBits will notexceed the upper limit or fall below this limit.FrameTargetBits=Min (FrameTargetBits, 0.8*FrameMaxBits, MaxTarget)FrameTargetBits=Max (FrameTargetBits, FrameMinTarget, MinTarget)The value of MaxTarget and MinTarget are defined as:MaxTarget [I frame]=GopTarget−(N−1)*(Number of Macroblocks per frame)*8MaxTarget [P or B frame]=GopTarget/NMinTarget [I frame]=GopTarget/NMinTarget [P or B frame]=(Number of Macroblocks per frame)*8F) Determine the Target Quantizer Scale for the Frame.

The target quantizer scale is calculated from the target frame bit rateand the complexity estimate, lower bounded to prevent incorrectcomplexity estimates from driving the quantizer scale too low. Thequantizer scale is also upper bounded by the maximum MPEG nonlinearquantizer scale value.FrameQuant=Max (MinFrameQuant, X[picture_(—)type]/FrameTargetBitsThe value MinFrameQuant is set to 2.

To improve stability in case of inaccurate complexity estimates,especially in complex still images, the algorithm sets a lower bound forthe target quantizer value of P and B frames if it drops suddenly from arelatively higher value. The following adjustments are made toFrameQuant if FrameQuant is smaller than the last AvgQuant value of thesame picture type and smaller than 24:FrameQuant=Min (StartupQuant, (FrameQuant+Max (LastFrameQuant,LastAvgQuant))/2where LastFrameQuant is the target quantizer (FrameQuant) value of themost recently coded frame of the same picture type, and LastAvgQuant isthe actual average quantizer (AvgQuant) value of the most recently codedframe of the same picture type.

The target quantizer scale (FrameQuant) value calculated on the MCC isthen clamped by a maximum value and delivered to the panel compressorsto be used initially to code the panels. Details for setting the upperbound and the quantization strategy on the panel compressors areprovided hereinafter. The quantizer scale values are adjustedperiodically at RC sync interrupts by a feedback process to drive theactual number of bits generated in the picture towards the target.

G) Delay Start of a New GOP if Buffer Level is too High

Before a new GOP may be started, the algorithm makes sure that there isenough room in the buffer to accommodate the surge of bits to begenerated by the upcoming I frame. Since the rate control task iscarried out after the decision of picture coding type, the buffer isinspected to determine the feasibility of starting a new GOP at the nextframe. If there is not enough space in the buffer, the rate control tasksends a signal to prevent the next frame from starting a new GOP. Thenext frame is allowed to start a new GOP if:EncoderBufferLevel+FrameTargetBits−(PictureDuration*BitRate)is below a threshold which is precomputed as:Min (0.5*BufferDelay*BitRate,TargetBufferLevel+(0.333*N*BitRate/FrameRate)5. MCC RC Sync Interrupt

Ideally, the frame target quantizer scale (FrameQuant) should be usedthroughout the picture to produce a uniform quality over the entirepicture. However, the rate control model based on the frame complexityestimate is not always accurate. Thus, adjustments to FrameQuant arenecessary to control the bit rate. A feedback mechanism is used suchthat the accumulated bit count of the frame is compared to the targetbit rate of the frame, scaled by the portion of macroblocks coded.

At RC Sync interrupt, the MCC reads two parameters from every panelcompressor. These parameters are (i) the number of macroblocks currentlyencoded, and (ii) the number of bits generated so far. The MCC sums upthese statistics for all eight panels (SumPanelBitCount andSumPanelMbCount). The MCC then computes a partial target and a partialmaximum for the number of bits generated.PartialFrameTargetBits=FrameTargetBits*SumPanelMbCount/(Number ofmacroblocks per frame)PartialFrameMaxBits=FrameMaxBits*SumPanelMbCount/(Number of macroblocksper frame)The adjustment of the quantizer scale is based on the ratios of theSumPanelBitCount to PartialFrameTargetBits and to PartialFrameMaxBits.If the bit count (SumPanelBitCount) exceeds 0.9 of PartialFrameMaxBits,the quantizer scale is adjusted as:CurrentQuant=CurrentQuant*0.9*SumPanelBitCount/PartialFrameMaxBitswhere CurrentQuant is the current quantizer scale being used, which isinitialized to the value FrameQuant at the start of encoding the frame.Otherwise the adjustment is based on the ratio of SumPanelBitCount toPartialFrameTargetBits using the following piecewise linear model:

ratio < 1.5 CurrentQuant = FrameQuant 2.0 ≧ ratio > 1.5 CurrentQuant =(1 + 0.5 * (ratio − 1.5)) * Max (FrameQuant, CurrentQuant) ratio ≧ 2.0CurrentQuant = 1.25 + (ratio − 2.0) * Max (FrameQuant, CurrentQuant)

Assuming that the bit count at the end of each frame is close to thetarget bit rate, the final quantizer scale value of the panel would beclose to the initial value which is FrameQuant. Such arrangement avoidsvisible seams at panel boundaries that can be caused by abrupt changesin the quantizer scale.

To improve stability, the value of CurrentQuant calculated in thecurrent RC sync is then averaged with the value of CurrentQuantcalculated in the previous RC sync.

The value of CurrentQuant is then capped to the same upper bound as thatused to cap FrameQuant (detailed hereinafter), and averaged with theprevious CurrentQuant value. Before sending the final value of quantizerscale to the panel compressors, CurrentQuant is further adjusted toincrease the quantizer scale value to protect the buffers if theSumPanelBitCount is approaching the limit of FrameMaxBits:if (SumPanelBitcount>0.975*FrameMaxBits) CurrentQuant=512else if (SumpanelBitCount>0.95*FrameMaxBits) AND(SumPanelMbCount<0.8*MacroblocksPerFrame)) CurrentQuant=4*CurrentQuantelse if (sumpanelBitcount>0.9*FrameMaxBits) AND(SumPanelMbCount<0.7*MacroblocksPerFrame)) CurrentQuant=2*CurrentQuantFinally, the value of CurrentQuant is capped again by the same upperbound as that used to cap FrameQuant. A lower bound of 2 also applies toCurrentQuant.6. Quantization Strategy

The panel compressors perform quantization by multiplying the DCTcoefficients with precomputed sets of reciprocal values of thequantizers. There are 32 precomputed sets of multipliers on the panelcompressors. The first 31 tables correspond to the MPEG quantizer scaleindex of 1 to 31, which are mapped to the quantizer scale value of 1 to112 according to the MPEG nonlinear quantizer mapping table. The 32^(nd)table is reserved for a panic mode in which all multipliers are set tozero, except for the intra DC, so that the DCT coefficients may bediscarded to minimize the number of bits generated. This panic modeshould be invoked only as an emergency measure to protect the buffers.The DC value of the non-intra panic multiplier table(PanicNonIntraDCMul) may be written by the MCC to determine whether tonormally quantize the non-intra DC by the maximum MPEG quantizer scale(112), or set it to a fraction of the normally quantized value. If thepanel compressor receives from the MCC a quantizer scale value(PanelQuant) between 1 to 112, the normal tables are selected. For anyPanelQuant value above 112, the panic table will be selected.

The quantization scale value calculated by the MCC (FrameQuant orCurrentQuant) spans a range of 1 to 512. This value is converted to an8-bit value from 1 to 255 (PanelQuant) to be sent to the panelcompressor to select the multiplier table, and is also used to determinethe non-intra DC multiplier of the panel compressor's panic table:

1 <= CurrentQuant <= 112: PanelQuant = round currentQuant to the nearestinteger by truncation. PanicNonIntraDCMul = MaxQuantNonIntraDCMul 112 <CurrentQuant <= MaxRcQuant: PanelQuant = 112; PanicNonIntraDCMul =MaxQuantNonIntraDCMul MaxRcQuant < CurrentQuant < 512: PanelQuant = 255;PanicNonIntraDCMul = MaxQuantNonIntraDCMul CurrentQuant = 512:PanelQuant = 255; PanicNonIntraDCMul = 0.3 * MaxQuantNonIntraDCMulwhere MaxQuantNonIntraDCMul is the non-intra DC entry of the multipliertable for the maximum MPEG quantizer scale (112). MaxRcQuant is aconstant value current set to 144.

Normally, the FrameQuant value should not be allowed to exceedMaxRcQuant above which the panel compressor may invoke panic mode thatseverely degrade the picture quality. However, if the encoder buffer isat the verge of overflowing, or if the decoder buffer is at the verge ofundeflowing, the panel compressors should be allowed to go into panicmode at the onset of encoding the frame. Therefore, the followingadaptive algorithm is used to set the upper bound of FrameQuant base onthe value of FrameMaxBits. If FrameMaxBits is big enough to support theworst case number of bits generated by normal MPEG quantization(FrameMaxBits exceeds a thresholdPanicThreshold1), FrameQuant is cappedby MaxRcQuant to avoid going into panic mode.

Once FrameMaxBits goes below PanicThreshold1, the upper bound ischanged, e.g., to 176, which enables panic mode on the panel compressorsbut preserves the non-intra DC coefficient at panic. If the FrameMaxBitsfurther drops below a second threshold PanicThreshold2, the upper boundis set to 512 so that the quantized value of the non-intra DCcoefficient can be reduced beyond that of normal quantization. Ahysteresis mechanism is provided so that once the upper bound is raisedabove MaxRcQuant, it will not return to MaxRcQuant until FrameMaxBitsexceeds a third threshold (PanicThreshold3). The thresholds{PanicThreshold1, PanicThreshold2, PanicThreshold3} can be set, forexample, to the values {750000, 500000, 1500000} for P and B frames, and{1500000, 1000000, 3000000} for I frames.

7. Panel Compressors

Each panel compressor receives the PanelQuant value from the MCC atevery RC Sync and at picture start. The panel compressors then performthe following routine on a macroblock by macroblock basis:

-   A) Update the encoding statistics.

Accumulate the macroblock count, PanelSumQuant (sum of actual quantizerscale used), and PanelBitCount (number of bits generated so far in thepanel). These statistics are reset to zero at the beginning of theframe.

-   B) Check fullness of local buffer; ramp up the quantizer scale if    the local buffer level is approaching full.    if (LocalBufferLevel<(112/128)*LocalBufferSize) PanelQuant unchanged    else if (LocalBufferLevel<(120/128)*LocalBufferSize)    PanelQuant=4*PanelQuant    else if (LocalBufferLevel<(124/128)*LocalBufferSize    PanelQuant=16*PanelQuant    else    PanelQuant=255-   C) Modulate PanelQuant by adaptive quantization the same way as in    MPEG Test Model 5.-   D) Apply the final quantizer scale to perform quantization.    8. Packet Processor

Whenever the video FIFO has at least one transport packet payload'sworth of data (184 bytes), the packet processor retrieves the data fromthe primary video FIFO at an average data rate equal to the video bitrate (bit_(—)rate) as specified by the user. Whenever the FIFO levelfalls below 184 bytes, the packet processor will stop retrieving datafrom the FIFO to protect the FIFO from underflow. Null packets will thenbe substituted for video packets to maintain a constant transport bitrate.

It should now be appreciated that the present invention provides a ratecontrol algorithm for optimizing the performance of a video encoder thatutilizes multiple parallel compression engines. The invention hasapplication to both HDTV and SDTV encoders.

Although the invention has been described in connection with variouspreferred embodiments, it should be appreciated that numerousadaptations and modifications thereof can be provided without departingfrom the scope of the invention as set forth in the claims.

1. A method for controlling the quantization in a digital video encoderthat comprises a plurality of parallel compression engines, comprisingthe steps of: determining a target quantization level for a video frame;wherein the video frame is represented by a plurality of panels, eachpanel comprises a plurality of slices, and each panel is processed inparallel by a respective one of the compression engines; encoding thefirst slice of each panel in accordance with said target quantizationlevel; and encoding subsequent slices in each panel in accordance with aquantization level that is allowed to vary from said target quantizationlevel until the last slice of each panel is reached; wherein thequantization level used for encoding the last slice of each panel isdriven toward said target quantization level.
 2. A method in accordancewith claim 1 wherein said driving step uses piecewise linear feedback todrive the quantization level of the last slice of each of said imagepanels toward said target quantization level.
 3. A method in accordancewith claim 2 wherein said feedback avoids abrupt variations in thequantization level between the first and last slice of each of saidimage panels.
 4. A method in accordance with claim 1 wherein a group ofpictures (GOP) target bit rate is adjusted based on a number of filmpictures and non-film pictures currently in a processing pipeline of atleast one of said compression engines.
 5. A method in accordance withclaim 4 wherein a higher target bit rate is provided for non-filmpictures.
 6. A method in accordance with claim 1, wherein: thequantization level used for encoding the last slice of each panel isdriven toward said target quantization level such that the first sliceand the last slice of each panel are encoded in accordance withapproximately the same quantization level.
 7. A method in accordancewith claim 1 wherein a buffer level of said video encoder is used tocontrol the start of a new group of pictures (GOP).
 8. A method inaccordance with claim 1 wherein said panels are simultaneouslycompressed at the respective compression engines during a frame time. 9.A method in accordance with claim 8 wherein the compressed panel dataare stored locally at the compression engines for subsequent transfer toa video buffer of the video encoder within a next frame time.
 10. Amethod in accordance with claim 9 wherein data are retrieved from saidbuffer, to form a transport packet, at an average rate equal to aspecified video bit rate whenever the buffer has at least one transportpacket payload's worth of data.
 11. A method in accordance with claim 10wherein null packets are substituted for video packets to maintain aconstant transport bit rate whenever said buffer level falls below onetransport packet payload's worth of data.
 12. A method in accordancewith claim 7 wherein a reference quantizer scale is calculated for eachof said compression engines.
 13. A method in accordance with claim 12wherein the reference quantizer scale for a compression engine iscalculated based on: the target quantization level, an accumulation ofquantizer scale values for each compression engine (sum_(—)quant), anaccumulation of the number of bits generated on each compression engine(bitcount); an accumulation of the number of macroblocks processed oneach compression engine (MBcount); and a fullness level of a videobuffer of the video encoder (buffer_(—)level).
 14. A method inaccordance with claim 13 wherein the compression engine modifies itsreference quantizer scale based on a local buffer fullness to generate afinal quantizer scale value for use in quantization.
 15. A method inaccordance with claim 14 wherein a panic mode is initiated by thecompression engine if the final quantizer scale value is higher than apredetermined maximum value, said panic mode maintaining thequantization at or below said predetermined maximum value.
 16. Apparatusfor controlling the quantization in a digital video encoder thatcomprises a plurality of parallel compression engines, comprising: meansfor determining a target quantization level for a video frame; whereinthe video frame is represented by a plurality of panels, each panelcomprises a plurality of slices, and each panel is processed in parallelby a respective one of the compression engines; means for encoding thefirst slice of each panel in accordance with said target quantizationlevel; and means for encoding subsequent slices in each panel inaccordance with a quantization level that is allowed to vary from saidtarget quantization level until the last slice of each panel is reached;and means for driving the quantization level at the last slice of eachof said image panels toward said target quantization level.
 17. A methodin accordance with claim 1, further comprising: computing said targetquantization level based on a complexity estimates from a number of pastframes, wherein: a complexity value of the most recently encoded I-frameis used as the complexity estimate for a current I-frame; an average ofthe complexity values of the four most recently encoded P-frames areused as the complexity estimate for a current P-frame; and an average ofthe complexity values of the four most recently encoded B-frames areused as the complexity estimate for a current B-frame.
 18. A method inaccordance with claim 17, wherein: a P complexity queue stores thecomplexity values of the four most recently encoded P-frames; and a Bcomplexity queue stores the complexity values of the four most recentlyencoded B-frames.
 19. A method in accordance with claim 18, wherein: inthe event that the current frame to be encoded is a first I-frame orP-frame after one of a scene change or a fade-in from black, the I-framecomplexity estimate and the complexity values stored in the P and Bcomplexity queues are conditionally replaced by corresponding I, P, andB default startup complexity estimates.
 20. A method in accordance withclaim 18, further comprising: comparing the stored values in the P and Bcomplexity queues and the I complexity estimate with the correspondingI, P, or B default startup complexity value; replacing the storedcomplexity value with the corresponding P and B default startupcomplexity value in the event that the stored complexity value is lessthan the corresponding default startup complexity value; replacing the Icomplexity estimate with the corresponding I default startup complexityvalue in the event that the I complexity estimate is less than thecorresponding default startup complexity value.
 21. A method inaccordance with claim 18, further providing: multiplying the storedcomplexity values of the P and B complexity queues with a constantgreater than 1 in order to increase the P and B complexity estimateswhere the current frame is a first frame after a fade-out to black hasbeen detected; multiplying the I complexity estimate by a constant lessthan 1 in order to reduce the I complexity estimate for every I frameduring a fade-out to black.
 22. A method in accordance with claim 9,wherein: the encoder buffer is a central buffer for the video encoder;and the buffer fullness level of the video encoder is provided by thesum of the levels of the encoder buffer and levels of local buffers ateach compression engine.
 23. A method in accordance with claim 22,further comprising: setting a target buffer fullness level at the startof every GOP.
 24. A method in accordance with claim 23, wherein: thebuffer fullness level is driven to the target buffer fullness levelusing feedback control at the start of every GOP.
 25. A method inaccordance with claim 17, further comprising: determining a targetnumber of bits to be generated during encoding of each group of pictures(GopTarget); determining a target number of bits to be generated duringencoding of each frame (FrameTargetBits); and allocating the GopTargetbits among each frame in the group of pictures in proportion to thecomplexity estimates for each frame.
 26. A method in accordance withclaim 25, further comprising: scaling the complexity estimates for eachframe by a corresponding weighting factor to account for differentperceptual and statistical characteristics between I, P, and B frames.27. A method in accordance with claim 25, further comprising: boundingthe FrameTargetBits to a fraction of a maximum allowable number of bitsfor the frame to avoid panic mode.
 28. A method in accordance with claim1, further comprising: setting a lower bound for the target quantizationlevel for P and B frames if the quantization level for the current P orB frame is lower than a previous quantization level for a correspondingP or B frame.
 29. A method in accordance with claim 1, furthercomprising: delaying the encoding of a new Group of Pictures (GOP) ifthere is insufficient space in an encoder buffer to accommodate an Iframe.
 30. A method in accordance with claim 1, further comprising:determining a maximum number of bits encoding of the current frame isallowed to generate (FrameMaxBits); initiating a panic mode in the eventthat FrameMaxBits falls below a threshold, said panic mode allowing agraceful degradation in video quality.
 31. A method in accordance withclaim 30, wherein: in the event that FrameMaxBits falls below a firstthreshold, non-intra DC coefficients are preserved during panic modeencoding, while all other DCT coefficients are dropped.
 32. A method inaccordance with claim 31, wherein: in the event that FrameMaxBits fallsbelow a second threshold which is lower than said first threshold, themagnitude of the non-intra DC coefficients is scaled to reduce themagnitude of the non-intra DC coefficients.
 33. A method in accordancewith claim 32, wherein: in the event that FrameMaxBits falls below athird threshold which is lower than said second threshold, all DCTcoefficients are dropped, including non-intra DC coefficients.